2. Probability Theory for Thermodynamics

Relevant readings and preparation

• Concepts in Thermal Physics: Chapter 3: 3.1-3.8: pg. 20-28

Learning outcomes:

• Distinguish between discrete and continuous probability distributions, and describe examples of each in physical contexts.

• Define and calculate the variance and standard deviation of a distribution.

• Understand how linear transformations affect the mean and variance of a random variable.

• Describe the properties and applications of the binomial distribution in modelling discrete physical events.

• Recognise the difference between independent and dependent probabilities, and compute joint or conditional probabilities accordingly.

• Apply Bayes’ theorem to update probabilities in light of new evidence.

• Identify and use the normal (Gaussian) distribution as an approximation to many natural phenomena.

• Describe the Poisson distribution and its relevance to counting statistics and random thermal events.

Reality is filled with uncertainty. Every action or decision we take must be made with incomplete information, since the chain of events leading to an outcome is often so complex that the exact result is unpredictable. Nevertheless, we can still act with quantifiable confidence in an uncertain world. Incomplete information is better than none at all, for example - it is more useful to know that there is a 20% chance of rain tomorrow than to have no forecast whatsoever. Probability is the mathematical framework that allows us to quantify uncertainty, and it is ubiquitous across all fields of scientific study, alongside finance sectors, software development, politics and so forth.

Probability theory has had an undeniably strong impact in furthering our understanding of thermal physics. This is because we often study systems containing a practically uncountable number of particles, where indiviual atomic behaviour is unpredictable but collective behaviour is remarkably regular. On macroscopic scales, probabilistic predictions become suitably precise. Measureable quantities such as temperature or pressure emerge as averages over many atomic contribtions. Although each atom behaves differently, and tracking all atoms’ individual motions and collisions is an unfeasible feat, the ensemble’s average behaviour follows well-defined probability distribitions, allowing us to perceive and model system behaviour without complete knowledge.

Before we delve into the basics of probability theory, we establish a few definitions:

• Probabilities are non-negative numbers which take values between 0 and 1.

• For a given scenario, all possible outcomes of that scenario form a set of events, with each outcome having an associated probability.

• If an outcome is not part of this set, its probability of occurring is zero.

• If the event is certain, the probability of it occurring is one.

• Events are considered ‘mutually exclusive’ if they cannot occur simultaneously.

• The sum of probabilities for all mutually exclusive outcomes must equal one for a valid probability distribution.

2.1. Discrete and Continuous probability distributions

Discrete Distributions

Discrete random variables can only take a finite number of values. The classic example of which is the humble six-sided die, which has a set of outcomes: {1, 2, 3, 4, 5, 6}. If we denote \(x\) a discrete random variable which takes values (i.e. outcomes) \(x_i\) with corresponding probability \(P_i\), we can make a few definitions which encapsulate the properties of discrete distributions.

First, we require that the sum of probabilities for every possible value of a discrete random variable equate to one:

\[ \sum_{i} P_{i} = 1 \]

we then define the arithmetic mean as the expected value of x, denoted:

\[ \langle x \rangle = \sum_i x_i P_i. \]

Intuitively, the idea is that for each possible outcome you adjust it’s contribution to the sum by its probability of occurance. This is called “weighting”. Were you to sample many many times, add all the outcomes you obtained and divide by the number of trials, you would eventually converge to the expected value. It is also possible to define the “mean squared” value of x through a similar procedure:

\[\langle x^2 \rangle = \sum_i x_i^2 P_i.\]

Do note that the expected value need not be present in the set of outcomes. A common example of this is the average number of children a family is expected to have across a population. These figures are often cited to occur between 1.8-2.4, yet it is only possible to have an integer number of children. These impossible values only make sense when considering a population rather than an individual sample.

Example: Expected value and mean squared

Consider a scenario where random variable \(x\) can take values {0, 1, 2} with corresponding probabilities {\(\frac{1}{2}\), \(\frac{1}{4}\), \(\frac{1}{4}\)}. This distribution is visualised in figure 2.1. Calculate the expected value for

• (a) the variable, \(\langle x \rangle\)

• (b) the mean squared of the variable, \(\langle x^2 \rangle\).

(a)

First check that \(\sum P_i = 1\). Since \(\frac{1}{2} + \frac{1}{4} + \frac{1}{4} = 1\) we are good to go. We then calculate the averages as follows:

\[\begin{split} \begin{align*} \langle x \rangle &= \sum_i x_i P_i \\ &= 0 \cdot \tfrac{1}{2} + 1 \cdot \tfrac{1}{4} + 2 \cdot \tfrac{1}{4} \\ &= \tfrac{3}{4} = 0.75 \end{align*} \end{split}\]

We see that the mean \(\langle x \rangle\) is not one of the possible values \(x\) can take.

(b)

We follow a similar process for \(\langle x^2 \rangle\):

\[\begin{split} \begin{align*} \langle x \rangle &= \sum_i x_i P_i \\ &= 0^2 \cdot \tfrac{1}{2} + 1^2 \cdot \tfrac{1}{4} + 2^2 \cdot \tfrac{1}{4} \\ &= 0 \cdot \tfrac{1}{2} + 1 \cdot \tfrac{1}{4} + 4 \cdot \tfrac{1}{4} \\ &= \tfrac{5}{4} = 1.25 \end{align*} \end{split}\]

Continuous distributions

Let \(x\) now be a continuous random variable, allowing it to take any value within specified bounds (these bounds could be infinite). We have to treat probabilities slightly differently here. Imagine a uniform distribution which, when sampled, can take any value between 1 and 10. One sample may yield an even 4, another something extremely specific such as 3.1415926535… Since there are infinite values that can be taken in this range, each individual value has effectively zero probability of occurring! Therefore, when calculating probabilities on continuous distributions we consider the probability of a variable having a value between some range \(x + dx\).

There are lots of real-life quantities that exist on continuous distributions. Height, commute durations, local temperature… These often have finite bounds, but there are infinite possibilities between them. Nonetheless, we enforce that the total probability of all values is one, however as we are now summing across continuous ranges we replace our sums with integrals:

\[\int P(x)dx = 1.\]

Similarly, we have analogous formulae for \(\langle x \rangle\) and \(\langle x^2 \rangle\):

\[\langle x \rangle = \int x P(x)dx;\]

\[\langle x^2 \rangle = \int x^2 P(x)dx.\]

Uniform Distribution on [0, 10]

Let a continuous random variable \(x\) be uniformly distributed between 0 and 10:

\[\begin{split} P(x) = \begin{cases} \frac{1}{10}, & 0 \le x \le 10, \\ 0, & \text{otherwise.} \end{cases} \end{split}\]

What is the values of

• (a) the expected value \(\langle x \rangle\)?

• (b) the mean squared value \(\langle x^2 \rangle\)?

First, ensure that \(P(x)\) is a valid probability distribution by checking that it is normalised:

\[ \int_0^{10} P(x) \, dx = \int_0^{10} \frac{1}{10} \, dx = \left[\frac{x}{10}\right]^{10}_{0} = 1. \]

We can then compute the expected values:

(a)

\[\begin{split} \begin{align*} \langle x \rangle &= \int_0^{10} x P(x) \, dx = \frac{1}{10} \int_0^{10} x \, dx = \frac{1}{10} \left[\frac{x^2}{2}\right]_0^{10} = 5, \\[6pt] \end{align*} \end{split}\]

(b)

\[ \begin{align*} \langle x^2 \rangle &= \int_0^{10} x^2 P(x) \, dx = \frac{1}{10} \int_0^{10} x^2 \, dx = \frac{1}{10} \left[\frac{x^3}{3}\right]_0^{10} = \frac{100}{3} \approx 33.33. \end{align*} \]

The mean of 5 lies exactly halfway between the bounds, which is as you’d expected for a symmetric uniform distribution.

Exponential Lifetime Distribution

Now consider a physical system whose lifetime follows an exponential distribution, such as the lifetime of a radioactive nucleus. The probability density of its lifetime is given by

\[\begin{split} P(t) = \begin{cases} \lambda e^{-\lambda t}, & t \ge 0, \\ 0, & t < 0, \end{cases} \end{split}\]

where \(\lambda\) is the decay rate constant. Calculate for this distribution:

• (a) \(\langle t \rangle\)

• (b) \(\langle t^2 \rangle\)

(a)

First, confirm normalisation:

\[ \int_0^{\infty} P(t) \, dt = \int_0^{\infty} \lambda e^{-\lambda t} \, dt = \lambda \left[-\frac{1}{\lambda} e^{-\lambda t}\right]^{\infty}_0 = \left[-e^{-\lambda t}\right]_0^{\infty} = 1. \]

Then compute the averages:

\[\begin{split} \begin{align*} \langle t \rangle &= \int_0^{\infty} t P(t) \, dt = \lambda \int_0^{\infty} t e^{-\lambda t} \, dt = \frac{1}{\lambda}, \\[6pt] \end{align*} \end{split}\]

(b)

\[ \begin{align*} \langle t^2 \rangle &= \int_0^{\infty} t^2 P(t) \, dt = \lambda \int_0^{\infty} t^2 e^{-\lambda t} \, dt = \frac{2}{\lambda^2}. \end{align*} \]

2.2. Measures of Central Tendencies

When describing a probability distribution, we often want to identify simple central values which best represent the properties and sampling behaviour of a probability distribution. These are known as measures of central tendency, and the most common are the mean, median, and mode:

• Mean ⟨x⟩: the expectation or average value of a distribution.

• Median: the value that divides the distribution into two equal halves, i,e. the middle value.

• Mode: the most probable value, where the probability is maximal.

For symmetric distributions (like the Gaussian), these three measures coincide onto the same value. For asymmetric or skewed distributions, they differ, and thus provide valuable information for characterising a distribution.

In skewed distributions, like in the positively skewed distribution in Figure 2.2, the mean is pulled towards the distribution’s tail whilst the median lies between the mean and mode. Regardless, the mode still represents the most probable value, i.e. the distribution’s peak.

Expectations of a function

The expectation value can be taken with respect to any arbitrary function, \(f(x)\):

\[\begin{split} \mathbb{E}[f(X)] = \begin{cases} \displaystyle \sum_i f(x_i)P(x_i), & \text{discrete}\\[6pt] \displaystyle \int_{-\infty}^{\infty} f(x)p(x)\,dx, & \text{continuous} \end{cases} \end{split}\]

Variance

The variance measures the average deviation of values around the mean of a distribution and is always positive. It is defined as follows:

\[ \mathrm{Var}(X) = \langle (x-⟨x⟩)^2 \rangle \]

We can expand the above expression to derive a simpler expression for calculating the variance of a scalar variable:

\[\begin{split} \begin{aligned} \mathrm{Var}(x) &= \langle (x - ⟨x⟩)^2 \rangle \\ &= \langle x^2 - 2⟨x⟩x + ⟨x⟩^2 \rangle \\ &= \langle x^2 \rangle - 2⟨x⟩⟨x⟩ + ⟨x⟩^2 \\ &= \langle x^2 \rangle - ⟨x⟩^2. \end{aligned}, \end{split}\]

which is often more practical for calculations by hand. This manner of expressing the variance gives rise to a mnemonic expression with which you can memorise the formula - The variance is “the mean of the squares minus the square of the means.” The variance is for a variable \(x\) is often denoted \(\sigma^2_x\), and relates to the standard deviation, which is simply the square root of the variance:

\[ \sigma_x = \sqrt{\mathbb{E}\left[(x- \mathbb{E}\left[x\right])^2\right]} = \mathbb{E}\left[x^2\right] - \mathbb{E}\left[x\right]^2 \]

2.3. Linear Transformations

A linear transformation is a mathematical rule which maps one variable onto another through a combination of scaling and shifting. For a scalar it takes the general form \(y = ax + b\), where \(a\) and \(b\) are constants:

• The constant \(a\) controls the scaling of the variable; how much the values are expanded or compressed.

• The constant \(b\) controls the translation of the variable; i.e. how far the values are moved up or down the axis.

In graphical terms, if \(x\) is represented on the horizontal axis and \(y\) on the vertical, the transformation \(y = a x + b\) produces a straight line whose slope is \(a\) and intercept is \(b\).

Example: Inches to centimetres

The conversion from inches to centimetres, or vice versa, is a linear transformation. If we take \(x\) to be inches, then to find the value \(y\) centimetres, we simply use the scale factor of \(a=2.54\) inches per centimeter and an intercept \(b=0\):

\[y\ \text{in} = x\ \text{cm} \times a + b = 2.54\ x\ \text{in} + 0\]

Expectation under a linear transformation

Because the expectation operator \(\langle \cdot \rangle\) is linear, its behaviour on a variable undergoing a linear transformation is simple:

\[ \langle y \rangle = \langle ax \ + \ b \rangle = a \langle x \rangle \ + \ b \]

This is because the expectation of a product is the product of expectations, and an expectation of sums is a sum of expectations. Broken down further:

\[\begin{split} \begin{aligned} \langle y \rangle &= \mathbb{E}\!\left[ a x + b \right] \\[4pt] &= \mathbb{E}\!\left[a x\right] + \mathbb{E}\!\left[b\right] \quad &&\text{(summation rule of expectation)} \\[4pt] &= \mathbb{E}\!\left[a\right]\mathbb{E}\!\left[x\right] + \mathbb{E}\!\left[b\right] \quad &&\text{(product rule for constants)} \\[4pt] &= a\,\mathbb{E}\!\left[x\right] + b \\[4pt] &= a\,\langle x \rangle + b \end{aligned} \end{split}\]

This is because the expected value of a constant parameter is merely itself. Thus it linearly warps the original expectation.

Variance under a linear transformation

Variance describes the spread of a distribution about its mean. For a variable \(x\) transformed linearly as \(y = a x + b\), we can derive how the variance changes.

Starting from the definition,

\[ \mathrm{Var}(y) = \mathbb{E}\left[(y - \langle y \rangle)^2\right] \]

and substituting \(y = a x + b\),

\[\begin{split} \begin{aligned} \mathrm{Var}(y) &= \mathbb{E}\!\left[(a x + b - \mathbb{E}[a x + b])^2\right] \\[4pt] &= \mathbb{E}\!\left[(a x + b - (a\,\mathbb{E}[x] + b))^2\right] \quad &&\text{(substitute } \mathbb{E}[a x + b] = a\,\mathbb{E}[x] + b\text{)} \\[4pt] &= \mathbb{E}\!\left[(a x + \cancel{b} - a\,\mathbb{E}[x] - \cancel{b})^2\right] \quad &&\text{(constants cancel)} \\[4pt] &= \mathbb{E}\!\left[(a(x - \mathbb{E}[x]))^2\right] \\[4pt] &= a^2\,\mathbb{E}\!\left[(x - \mathbb{E}[x])^2\right] \quad &&\text{(constant factor rule)} \\[4pt] &= a^2\,\mathrm{Var}(x) \end{aligned} \end{split}\]

The additive constant \(b\) cancels out because it merely shifts the entire distribution without changing its spread. The multiplicative constant \(a\) scales the variance by \(a^2\), since stretching or compressing the variable by \(a\) changes the width of the distribution quadratically. These relationships underpin how measurement uncertainty propagates through unit conversions or calibration - when a physical quantity is scaled by \(a\), its variance (and hence uncertainty) scales by \(a^2\).

2.4. Independent Variables in Probability

In many physical problems we deal with more than one random variable. For example, the position and velocity of a molecule, or the outcomes of repeated measurements of the same quantity. Sometimes these values are independent of one another, meaning that knowledge of one variable provides no information about the other. A standard example of this is coin flips, knowing the outcome of one coin flip provides no picture on what the necxt flip will yield.

Formally, two random variables \(u\) and \(v\) are said to be independent if their joint probability distributed can be written as a product of their indovividual distributions:

\[ P(u, \ v) = P_u(u) \times P_v(v), \]

thus the value taken by \(u\) does not depend on \(v\) and vice versa. This extends for any amount of independent variables, i,e. \(\vec{x} \in \mathbb{R}^N\) is described by:

\[ P(\vec{x}) = \prod_{i=1}^N P_i(x_i) = P_1(x_1) \times P_2(x_2) \ \times \ ... \ \times P_N(x_N). \]

A useful consequence of independence is that the expectation value of a product of indepedent variables factorises neatly:

\[\begin{split} \begin{align} \mathbb{E}\!\left[uv\right] &= \int \!\!\int uv \, P_u(u)P_v(v) \, du\,dv \\[6pt] &= \left(\int u \, P_u(u) \, du\right) \left(\int v \, P_v(v) \, dv\right) \\[6pt] &= \mathbb{E}\!\left[u\right] \, \mathbb{E}\!\left[v\right]. \end{align} \end{split}\]

We can achieve this because integrals for indepedent variables can be separated. This implies that the average value for the product of \(u\) and \(v\) is equal to the product of their average values. Again this can be extended to any arbitrary number of independent variables:

\[\begin{split} \begin{align} \mathbb{E}\!\left[\prod_{i=1}^{N} X_i\right] &= \int \cdots \int \Bigg(\prod_{i=1}^{N} x_i\Bigg) \Bigg(\prod_{i=1}^{N} p_i(x_i)\Bigg) \, dx_1\cdots dx_N \\[6pt] &= \prod_{i=1}^{N} \left(\int x_i\, p_i(x_i)\, dx_i\right) \\[6pt] &= \prod_{i=1}^{N} \mathbb{E}\!\left[X_i\right]. \end{align} \end{split}\]

The same is true for discrete variables, or even a combination of discrete and continuous variables. Simply replace integral signs with sums for discrete components, but otherwise the product of expectation values is equivalent for these combinations of variable type.

Example: Mean and variance

Suppose that there are \(n\) independent variables, \(X_i\), each with the same mean \(\langle X \rangle\) and variance \(\sigma^2_X\). Let \(Y\) be the sum of the random variables, such that \(Y = X_1 + X_2 \ + \ ... \ + \ X_n\). Find

• (a) the mean of Y

• (b) the variance of Y

(a)

The mean of \(Y\) is simply the sum of each variable’s expectation value. As each variable \(X_i\) has the same mean \(\langle X \rangle\), we have:

\[ \langle Y \rangle = n \langle X \rangle. \]

(b)

Finding the variance of \(Y\) is a marginally more complicated matter. To start, let’s refer to the formula \(\sigma^2_Y = \langle Y^2 \rangle - \langle Y \rangle^2\). Seeing as we have \(\langle Y \rangle^2 = n^2 \langle X \rangle^2\), we only need to calculate \(\langle Y^2 \rangle\):

\[\begin{split} \begin{align} \langle Y^{2} \rangle &= \langle X_{1}^{2} + \cdots + X_{N}^{2} + X_{1}X_{2} + X_{2}X_{1} + X_{1}X_{3} + \cdots \rangle \tag{3.37} \\[6pt] &= \langle X_{1}^{2} \rangle + \cdots + \langle X_{N}^{2} \rangle + \langle X_{1}X_{2} \rangle + \langle X_{2}X_{1} \rangle + \langle X_{1}X_{3} \rangle + \cdots \end{align} \end{split}\]

There are \(n\) terms like \(\langle X_1^2 \rangle\) on the right-hand side, and n(n-1) terms like \(\langle X_1 X_2 \rangle\). The former takes the value \(\langle X^2 \rangle\) and the latter \(\langle X \rangle \langle X \rangle = \langle X \rangle^2\). Therefore:

\[ \langle Y^{2} \rangle = n\langle X^2 \rangle + n(n-1)\langle X \rangle^2, \]

and,

\[\begin{split} \begin{align} \sigma_y^2 &= \langle Y^2 \rangle - \langle Y \rangle^2 \\[6pt] &= n\langle X^2 \rangle + n(n-1)\langle X \rangle^2 - n^2\langle X \rangle^2 \\[6pt] &= n\langle X^2 \rangle + n^2\langle X \rangle^2 - n\langle X \rangle^2 - n^2\langle X \rangle^2 \\[6pt] &= n\langle X^2 \rangle - n\langle X \rangle^2 \\[6pt] &= n\!\left(\langle X^2 \rangle - \langle X \rangle^2\right) \\[6pt] &= n\sigma_X^2 \end{align} \end{split}\]

This tells us that if we make \(n\) independent measurements of the same quantity, and then take their average via \(Y/n\), the uncertainty in that average is reduced by a factor \(\sqrt{n}\) compared to a single measurement:

\[ \sigma_{mean} = \frac{\sigma_x}{n}. \]

TODO - RMS needs elaboration, cite random walks and possibly Brownian motion.

This principle - that averaging many independent measurements reduces random error - underlies much of experimental physics. However, it is only applicable for random, uncorrelated errors. Any systematic bias in a measurement setup will persist regardless of repetitions. A related idea actually appears in the study of random walks; such as the motion of a particle buffeted by molecules in a fluid. Each step (period of motion) is independent of the last, so whilst the average displacement after many steps is zero, the root-mean-square displacement grows as \(sqrt{n}\).

2.5. The Binomial Distribution

Consider an experiment with only two possible outcomes, success and failure. This is called a Bernoulli trial.

• Let the probability of success be p.

• Then the probability of failure is (1 − p).

If we assign the value 1 to a success and 0 to a failure, the expectation and variance of a single trial are:

\[ \langle x \rangle = p, \qquad \sigma_x^2 = p(1 - p) \]

Now imagine performing \(n\) independent Bernoulli trials; for example, flipping a coin \(n\) times where we count \(k\) heads. Let us consider “heads” as a success.

There are two ingredients to the probability of obtaining k successes:

1. The probability of one specific sequence with k successes and n − k failures:

\[ p^k (1 - p)^{n - k} \]

2. The number of possible sequences with exactly k successes:

\[ {n \choose k} \]

Multiplying these gives the binomial probability of observing k successes given n trials:

\[ P(n, k) = {n \choose k} p^k (1 - p)^{n - k} \]

---

2.6. Properties

From the definition, one can show that the probability of all combinations of successes and failures sums to unity:

\[ \sum_{k=0}^{n} P(n, k) = 1, \]

so the probabilities are properly normalised. The mean and variance of k are:

\[ \langle k \rangle = np, \qquad \sigma_k^2 = np(1 - p) \]

As n increases, both the mean and standard deviation grow, but the fractional width

\[ \frac{\sigma_k}{\langle k \rangle} = \sqrt{\frac{1 - p}{np}} \]

decreases, meaning the distribution becomes more sharply peaked around k = np.

---

Example - coin tossing

For a fair coin, \(p = \frac{1}{2}\).

• For 16 tosses, expected heads: \(\langle k \rangle = 8\); \(\sigma = 2\).

• For \(10^{20}\) tosses, expected heads: \(\langle k \rangle = 5\times10^{19}\); \(\sigma = 5\times10^{9}\) - ten orders of magnitude smaller relative to the mean.

Thus, as the number of trials increases, the relative fluctuations around the mean gradually become negligible.

Example: 1D random walk

A one-dimensional random walk can be viewed as successive Bernoulli trials: each step is either forward (+L) or backward (−L) with equal probability \(p = \frac{1}{2}\).

After n steps, if k are forward, the net displacement is

\[ x = (2k - n)L. \]

Using the binomial results,

\[ \langle x \rangle = 0, \qquad \langle x^2 \rangle = nL^2, \qquad \sigma_x = \sqrt{n}\, L, \]

showing that the root-mean-square displacement grows as \(\sqrt{n}\) - a key result that later links to Brownian motion.

2.7. Conditional Probabilities - Bayes’ Theorem

In many physical problems, events are not independent. More often than not, the probability of one event can depend on whether another has occurred. The framework in which we describe dependent events is called “conditional probability”.

---

Conditional probability

If we have two events, A and B, the probability of A given that B has occurred is written:

\[ P(A\, |\, B) = \frac{P(A \cap B)}{P(B)} \]

provided \(P(B) \neq 0\). Here, \(P(A \cap B)\) means the probability that both A and B occur.

When events are independent, the occurrence of B does not affect A, so

\[ P(A\, |\, B) = P(A) \]

and equivalently \(P(A \cap B) = P(A)P(B)\).

---

Bayes’ theorem

Rearranging the definition of conditional probability gives an extremely useful result known as Bayes’ theorem:

\[ P(A\, |\, B) = \frac{P(B\, |\, A)\, P(A)}{P(B)} \]

This allows us to update our belief about A after observing B. In other words, it connects the prior probability \(P(A)\) with the posterior probability \(P(A|B)\).

Example: medical testing

A common example used to understand Bayes’ theorem is testing for disease. Suppose a disease affects 1% of a population. A test correctly identifies it 99% of the time, but also gives a 5% false-positive rate. In other words, if a person has the disease, the test will return a positive result 99% of the time, and a negative result 1% of the time. Meanwhile, if the person does not have the disease, it will return a negative result 95% of the time, and a positive result 5% of the time.

Let A = “person has disease” and B = “test is positive”. Then:

• \(P(A) = 0.01\)

• \(P(B|A) = 0.99\)

• \(P(B|\neg A) = 0. 05\)

The overall probability of obtaining a positive test result must factor in both means of obtaining that result:

\[ P(B) = P(B|A)P(A) + P(B|\neg A)P(\neg A) = 0. 99(0. 01) + 0. 05(0. 99) = 0. 0594 \]

Applying Bayes’ theorem:

\[ P(A|B) = \frac{P(B|A)P(A)}{P(B)} = \frac{0. 99 \times 0. 01}{0. 0594} \approx 0. 17 \]

So even with a positive result, there’s only a 17% chance the person actually has the disease.

This illustrates how rare events can strongly influence conditional probabilities. Follow-up testing can help in reducing the uncertainty of a result, by tightening the variance over repeated trials. That is to say, the probability of obtaining several positive test results in a row, given that the person does not have the disease, is astoundingly small, (and is likely indicative that the person does in fact have the disease.)

2.8. Key Distributions - Guassian, Poisson, Maxwell-Boltzmann

Gaussian (Normal) Distribution

A continuous, bell-shaped probability density function:

\[ f(x; \mu, \sigma)=\frac{1}{\sqrt{2\pi}\, \sigma}\exp\! \left(-\frac{(x-\mu)^2}{2\sigma^2}\right) \]

The mean and variance are \(\langle x\rangle=\mu\) and \(\sigma^2\). By the Central Limit Theorem, the sum or average of many independent random variables tends toward a Gaussian distribution, regardless of the original shape.

---

2.9. Poisson Distribution

A discrete distribution describing the number of rare, independent events occurring in a fixed interval:

\[ P(k; \lambda)=\frac{\lambda^k e^{-\lambda}}{k! }, \qquad k=0, 1, 2, \dots \]

The mean and variance are both \(\langle k\rangle=\lambda\). It arises as the limit of the binomial distribution for large \(n\), small \(p\), with \(\lambda = np\).

---

2.10. Maxwell–Boltzmann Speed Distribution (3D Ideal Gas)

The Maxwell-Boltzmann distribution is a particular probability distribution named after James Clerk Maxwell and Ludwig Boltzmann, and was first defined for describing particle speeds in idealized gases.

\[ f(v; T, m)=4\pi\! \left(\frac{m}{2\pi k_B T}\right)^{3/2} v^2 \exp\! \left(-\frac{m v^2}{2k_B T}\right), \qquad v\ge 0 \]

Key quantities:

• Most probable speed: \(v_p=\sqrt{\frac{2k_B T}{m}}\)

• Mean speed: \(\langle v\rangle=\sqrt{\frac{8k_B T}{\pi m}}\)

• RMS speed: \(v_{\mathrm{rms}}=\sqrt{\frac{3k_B T}{m}}\)